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BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of 
the original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 

• BLACK BORDERS 

• TEXT CUT OFF AT TOP, BOTTOM OR SIDES 

• FADED TEXT 

• ILLEGIBLE TEXT 

• SKEWED/SLANTED IMAGES 

• COLORED PHOTOS 

• BLACK OR VERY BLACK AND WHITE DARK PHOTOS 

• GRAY SCALE DOCUMENTS 

IMAGES ARE BEST AVAILABLE COPY. 



As rescanning documents will not correct images, 
please do not report the images to the 
Image Problem Mailbox. 
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Initialize CS and/or CC for 
theme set for user 
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I aTcS, obtain data signals from input devices ^ 0 ^ 



At CS, process data signals to produce attributes j 

representing user's context, \^ 

including adding required data fields, ( 

such as attribute name, timestamp, units, etc. (' 
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aTcS, provide attribute to CM for storage ^ 0 8 



| At CC, request from CM and receive attributes V \, M- ^ 0 



CC, procesTattributes to determine appropriate output M" ' 2^ 



■ Provide output signal to user, other entity or other process L| ! ^ 



Code 
Module A 



Code 
Module B 

V (CC) ^ 



Code 
Module C 



Code 
Module D 



CM 




Launch at start-up 



'-~-^=£: |^Periodi c^l ^ GPS data streamj q — ^ 



Parse and store GPS data 





Receive request ]vj 
for latitude, longitude or J>^— 
altitude attribute? 
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Provide requested attribute value(s), with 
Timestamp, uncertai nty, units, etc. 

-_] ^ 



Latitude fVcLt) 
Uncertainty 



Timestamp 



Units 



Format Version/Flags 




j ^egister for latitude, longitude and altitude attributes 




Normal operation? 
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Use default attribute mediator 60 6 



Reevaluate all latitude attributes 




[selec t lowest uncertainty attribute m ediator ^ 

ZlZi 

i Determine CS providing resulting attributes ^ ^ ^ ^ 
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Request latitude, longitude and altitude attributes from CS/O^}^ ^' ^ 
— Notify feature active? > 
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Receive input defining boundary~"7^ ^ ^ ^ 
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[Provide e vent paramete rs to CM regarding boundary transversal { 



Receive response froiiT 
!M to established eventZ,^^ K) 



pJotify user that boundary has been crossed^ 
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User. 

Desired_privac>'_level 

Intemiptibility 

Speed 

Direction 

Acceleration 

Availability. 

Cognitive_availabilit>' 

Tactile_availabilit>' 

Manual_availabilit>' 

Visual_availability 

Oralavailability 

Auditory_availabilitY 
Proximity. <Item or place name> 
Mood. 

Happiness 

Sadness 

Anger 

Frustration 

Confusion 
Activity. 

Driving 

Eating 

Running 

Sleeping 

Talking 

Typing 

Walking 
Location. 

Place_name 

Latitude 

Longitude 

Altitude 

Room 

Floor 

Building 

Address 

Street 

City 

County 

State 

Country' 

Postal_Code 
Destination, (same as User.Location.) 
Physiolog>\ 

Pulse 

Body_temperature 

Blood_pressure 

Respiration 
Person.<name or ID>. (same as User.) 
Platform. 
UI. 

Oral_input_de\ace_availabilit>' 

Manual_input_device_availability 

Tactile_outpul_device_availability 

Visual_output_device_availability 

Auditory_output_device_availability 



Platform, (continued) 
CPU. 

Load 

Speed 
Memor}'. 

Total_capacit>' 

Used 
Storage. 

Total_capacity 

Used 
Connectivit>'. 

Connectionstatus 

Connection_speed 

Connection_type/device 

Connectionactivity 
Power, 

Power_source 

Powerlevel 
Environment. 
People, 

Nearest 

Number_present 
Number_close 
Local. 
Time 
Date 

Temperature 

Pressure 

Wind_speed 

Wind_direction 

Absolute_humidit>^ 

High_forecast_temperature 

Low_forecast_temperature 

People_present 

Ambient_noise_level 

Ambient_light_level 

Days.<previous or ftiture>. 

Hightemperature 

Lowjemperature 

Precipitationtype 

Precipitationamount 
Place.<place name>. (same as Environment. Local) 
Application. 
Mail. 

Available 

New_messages_waiting 
Messages_waiting_to_be_sent 
Phone. 

Available 

ln_use 

On/oflf 

Notification_mechanism 
Call_incoming 
Caller_ID 
Sound^recorder, 
Available 
Recording 
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User Setting 

Mental Context 
Meaning 
Cognition 

Divided User Attention 

Task Switching 

Background Awareness 
Solitude 
Privacy 

Desired Privacy 

Perceived Privacy 
Social Context 
Affect 
Physical Situation 
Body 

Biometrics 

Posture 

Motion 

Physical Encumberment 
Senses 
Eyes 
Ears 
Tactile 
Hands 
Nose 
Tongue 
Workload demands/effects 
Interaction with computer devices 
Interaction with people 
Physical Health 
Environment 
Time/Space 
Objects 
Persons 

Audience/Privacy Availability 
Scope of Disclosure 
Hardware aflRnity for privac>' 
Privacy Indicator for User 
Privacy Indicator for Public 
Watching Indicator 
Being Observed Indicator 
Ambient Interference 
Visual 
Audio 
Tactile 



Computer 

Power 

Configuration 

User Input Systems 
Hand/Haptic 

Keyboard/Keystrokes 
Voice/Audio 
Eye Tracking 
Cursors 
Axis 

Resolution 
Selection 
Invocation 
Accelerators 
Output Systems 
Visual 

Resolution 
Audio 

Public/Private 
Haptic 
External Resources 
I/O devices 
Connectivity 

Data 

Quantity/State 
Urgency/Importance 

Use of Prominence 
Modality 

Sensitivity/Purpose 
Privacy Issues 
Use of Association 
Use of Safety 
Source/Ownership 
Types 

User generated 

Other computers or people 

Sensor 

PC State 
Use of Association 
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School Shooting Near Son Diego Leaves Two 
Dead, Afora Titan a Dozen Wounded 

Northeast U.S. Waits for Worst of Snowstorm; 
SchooiSf Airports Disrupted 



France Finds Foot^arkhMouth Antibodies in 
Sheep; Animal Trartsport Curbed 



wssen to remain failed until his trial '-^ 

1 'Woc^JstM'' ■M:;:.y:4k!^ 



Inside Temperature: 70 
Outside Tempemture: 36 
Wind:WNW@Smph 
Currently windy 




Tell Dan about Tech Talk 
Remind DavidB about Design . 
Fbc DavidB's Computer 
Find kennel for dog - 
Take out the garbage 
Buy Bread and Milk 
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10:00 Company meeting 



11(5 




The Wife . 
The Kid 



11/7 



lift 
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Tengh 

1848 Westiake Ave N 
Seatth, WA 
47.626 N '122.34W 




Tora XXXrX onto M13Z\AArxr STC. 
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70:00 Company Meeting 



PeoptetpCall 



John r(42S)m-S5S5LOf 
Mark Smith (206)936^214 SDE Offer 
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«. Arriving Al Work 



Ancei Johnson 



TtUDan about Tech Talk 



Remind DavldB about Design spec deadline. 
Fix DavidB's Computer 

Call Mark Smith (206/936-6214 about Job Offer 



>tang|Bi Biiijc^n Board' 
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1 Weekly Company meeting at 10:00pm 

2 VIA demo at 3:00pm Penthouse. Everyone invited. 

3 Wito let my dog out?[Dan] 

4 Extra ticlcet to Sonics/Lalcers game tonight[Josh] 

5 Who used decaf on the non decaf container?l?[Aian] 
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AtmbrkThemi^ 



iCatendar 



JbDotLlrt 













:r:^jOb[|'A^ Conpany Meeting 
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fe// Oa/y aboi/^ Tech Tatk 



Remind DavidB about Design spec deadth \ 
Fix Da¥idB*s Computer 
See Bin about networking problems 
CaHi\/fark Smith (20e)9^&S2U about SDE I 



Bill Watson 
Alan Housley 
Dorothy B. 



Flo. (ID 
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T II group what V u did last we k 



Tell group what you plan to do this week 
Tell group about your new Invention. 
Tell group about your upcoming skiing trip. 
Kudos for Stan and Dorothy for finishing report 



Notes i 



Finished coding Turing Machine on Monday. 
Did bug bash on Tuesday. 
Survived an earthquake on Wednesday. 
Worked on CTK demo rest of last week 
Continuing work on CTK demo rest of this week 



Aaron Graves 
Alan Housley 
An eel Johnson 
Bill Watson 
Brian Rose 
Cesar Ahfarez 
Dan Newell 
David Jukes 
Dorothy B. 
Jaqule T. 
Jeff Pried 
Jim Robarts 
Johnson A. 
Josh Freodman 
Kanisha Patel 
Ken Abott 
Lisa Davis 
Molly Fitch 
Scott Price 
Stan Emert 
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Me owes you moo 
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, Lunch Willi Oan 



Restaurant In the Area 



Kwanjai Thai 
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Thai Restaurant 
1800 WesUakB Ave 



Vs. 



Prevfdiis:: 




i^taruals' Empl<n/«i»;.C#iilne 



Gnat Thai food. Highly Recommended. -Josh 
Pretty good thai food. Try oat their panang. -Johnson 



Rcpdrifig a PC 
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Dell XPS 4(mi) 

Purchase Date: 11'20/2000 

TanrjisTag: PCOOI 









Office 2000 ^ 

V'tsual Studios 6.0 

MS Entertainment Pack 

Boeing Marketing Tutorial: How to 

talk like a techie ^ 








Creative Sound Blaster 10.0 card ii^ 

Ricoh 32XCDRW 

MS USB Microphone 

Intel TR1688 100Mbps Token Ring 



{Currently inoperative 
j Josh noticed a loud noise coming from 
the hard disk. 

He said h might be a problem wHh the 
disk's brake pads. 

If he mentions this again, humor him. 
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Driving Home 



Your wife is at Nordstrams. 



Current house temperature: S5F 



Tod 
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Find kennef for dot 
Bring in the garbac 
Buy Bread and Mil 
Buy CC ice Cream 




WARNING!!! 

SUDDEN STOP 
IMMINENT 



lift »•) --Ml' 



; after worif. 
tfne CC ice cream. 



Findkenn I for (fog. 
Bring in th garhagB. 
Help Junior with Homework. 



07.-00 Komo TV Local News (4) 
07:30 Wall Street Week (9) 
OtM Survivor a (7) 
09.-00 X-Flies (13) 
10:00 Big Apple (7) 
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! Family; Boa^ 



IS poiiseUM I am ooin Q to Nordstro ms after work. 



[SonJ4:40 Dad, Can you buy me some CC ice cream, 
lSpovseJS:U 111 be home at around 6:30. Get dinner. 
[fouJS:15 Sure Son. 
{Youjd:00 I'm home. 



The Wife 
The Kid 
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Example Current Theme Selection 
Select A New Current Theme 




Starting The Day 

Driving Home 
At Home 

Work-Related Group 
Driving To Work 
Arriving At Work 
At Work 

Entertainment-Related Group 





Starting The Day 

At Home 
Eating Breakfast 



Figure IIM 

Example Current Theme Set Modification 
Current Theme Set 



Add Theme 



Remove Theme 



Figure UN 



Example Current Theme Layout Selection 
Select A New Current Theme Layout 

Starting The Day theme 
/- Starting The Day - default layout 

y' — Starting The Day - vacation 

^ Starting The Day - responsible for dropping off kids 



OK 



Cancel 
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Figure HO 



Setconte)^ 



PageUp ^ 

# Choices: 

1-^1 New 

;;??2 Organize 
i;p3 Set Theme 




^. Coiilexl_Clrl 



# Choices: 

;::;;^2 Organize 
;5;!3 Set Theme 
iEt4 Me 
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CurUeKl Clrl 
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rSet context Jnfprm^^^^^^ 



"'T PageUp ^ 
# Choices: 



,1 New 



;i;:Sr2 Organize 
i¥v;-^3 Set Theme 

r; mA Me 



ipcaflbri |j|m^ office ~^ I 
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#Choices: 
,i New 

:2 Organize 



Objects ii-ii 



ry 
Id 



! if 



c:5 



;;:;:3 set Theme 
> ;f:r4 Me 



I Activity 




People 

•js7 Places 
, [if 3 Environment 



Oiitstde : i 36 



Objects 



It is [windy ^ today ^ 



Page Down ▼ 
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. b|2 Organize 
^3 Set Theme 

'L!4 Me 



I Activity 



People 
^:J7 Places 
• Environment 



Objects 



Dan^ Computer 
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Xybemaut MA IV 



Set Focus 



Page Down ▼ 
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^. Coiilexl Ctrl 



■( Page Up ^ 

# Choices: 
New 

;i-sf2 Organize 

-r?^3 Set Theme 

'■:'-:-"t4 Me 



Activity 



People 
1 ;G7 Places 
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Environment 



•r^Jglg Objects 



Page Down ▼ 
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Example Theme Data Structure 
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Name 



Driving To Work 



Description 



Intent Is To Organize Both Driving Info & Info About Work Arrival 



Security 



Permission 



(33© 



Privacy 



Priority 



Medium 



Group-ID 



23 



f3VO -^ime- Active 



Mon-Fri, 06:00-11:00 



Source 



BCD Motor Company 



1 3 t^T ^heme-Content 



'BCD Motor logo.gif ' 



^-^ ^ 'New- Attributes 
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DTW-BCD.driving-direction: (source = "BCD-DTW CS"); 



hi 



Theme- 
Matching 



user. activity. driving = true (required); environment.local.time = 08:00-09:00 
(optional); DTW-BCD.driving-direction = NOT "South" (required); . . . 



Theme-Logic 



IF <user. activity. driving> AND user. location. latitude > recent.user. 

location.latitude THEN DTW-BCD.driving-direction = "North"; 
IF DTW-BCD.driving-direction = "East" THEN Alert("You're Lost Again!!"); 
SET user. location[ Access] - Family & Co-workers; . . . 



3->oC;t^ Attribute-Set 




Attribute J 




Attribute N 



Name 


user, activity, driving 


Value 


True 


Uncertainty 


High 


Timestamp 


07:32 03/24/XX 


Units 


boolean 


Access 


Family 


• 



Name 


DTW-BCD.driving-direction 


Value 


West 


Uncertainty 


None 


Timestamp 


08:13 03/24/XX 


Units 


North, North by Northeast, . . . 


Access 


Self 


• 
• 



Figure 13 
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-Verbal 

-GPS ► 



Mouse — 
Keyboard— 
Time ► 



-Message Loop*^ 



Fitters & 
Builds Datastore 



Actions 

Modify 
Windows 



Manage PIM 



Initiate calls, 
pages, faxes, 
email, web 
searchs 



Display 
Maps 



more... 




Preference 
Pattern 




Pattern Recognition 



Inference Engine 



Rules 
Time 



Place 



Command 



File 



Learning 
Preferences 



User Defined 
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